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\ SUMMARY 

The  application  of  optimization  techniques  to  the  derivation  of  predictive  information 
for  flight  displays  is  being  investigated  in  connection  with  operational  situations  involving 
large  disturbances  and  manoeuvres.  This  note  surveys  current  gradient  methods  for 
computing  extremal  solutions  of  optimal  control  problems  for  systems  having  boundary 
conditions  but  without  state  or  control  constraints.  It  is  concluded  that  a two  stage 
procedure  is  required,  with  the  first  stage  using  a first  order  gradient,  while  the  second 
stage  would  use  a higher  order  variable  metric  method.  Proposals  are  advanced  for 
further  research  on  optimization  algorithms. 
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1.  INTRODUCTION 

The  extension  of  flight  director  capabilities  to  a wide  range  of  operational  situations  including 
major  manoeuvres  and  disturbances  is  becoming  feasible  through  advances  in  digital  avionics 
and  displays.  This  will  permit  the  timely  presentation  of  alternative  strategies  to  the  aircrew,  for 
their  use  in  decision  and  control.  These  types  of  predictive  aids,  which  for  example  would  be 
useful  for  STOL  aircraft  landing  at  forward  tactical  airfields,  and  for  combat  aircraft  involved 
in  tactical  engagements,  are  receiving  increased  attention  in  the  United  States  and  NATO 
countries.1  5 Research  work  is  being  initiated  at  ARL  to  gain  an  improved  understanding  of  the 
potentialities  and  limitations  of  these  systems. 

The  determination  of  the  optimal  manoeuvres  required  to  achieve  specific  objectives,  whilst 
taking  into  account  physical  and  operational  constraints,  would  enhance  the  value  of  a director 
under  normal  operating  conditions,  and  would  be  of  especial  value  in  emergency  situations. 
This  note  surveys  current  methods  used  for  computing  extremal  solutions  of  optimal  control 
problems  for  systems  having  boundary  constraints,  but  without  state  or  control  constraints. 
The  extension  to  the  general  case  with  constraints  through  penalty  function  and  other  tech- 
niques will  be  the  subject  of  further  work. 

Particular  attention  is  focused  on  the  so-called  “direct  methods”  and  as  many  of  these 
methods  cannot  be  immediately  applied  to  problems  having  terminal  constraints,  a discussion 
of  techniques  for  adapting  them  to  this  need  is  presented.  Finally  a nunjber  of  approaches  not 
studied  in  the  literature,  which  are  worthy  of  further  examination  will  be  briefly  considered. 


2.  ALGORITHMS  FOR  UNCONSTRAINED  PROBLEMS 


Computational  algorithms  for  seeking  the  extremal  solution  of  general  optimal  control 
problems  fall  into  two  main  groups,  which  are  known  as 

(a)  Indirect  Methods,  and 

( b ) Direct  Methods. 


In  addition  to  the  methods  mentioned  above,  special  algorithms  for  finding  the  optimal 
control  of  systems  described  by  linear  differential  equations  have  also  been  extensively  developed. 
As  the  equations  of  motion  for  the  flight  mechanics  problems  of  interest  are  generally  non- 
linear these  methods  are  inapplicable  and  so  will  not  be  considered  further. 

The  flight  mechanics  optimization  problems  being  considered  can  be  formulated  within  the 
framework  of  the  general  optimization  problem  of  Bolza:  for  a fixed  final  time,  If,  find  an  un- 
bounded control  function  u{t),  1 e [t0,  //]  which  minimizes  the  cost  functional 


J(u)  = <)>[x(t/),  tf]  + 


L(  x,  u , t)dt 


(2.1) 


subject  to  the  differential  equation 


d*  (l)  = /(•*.  u,  l),  x(t„)  = .x„ 

at 


(2.2) 


and  terminal  conditions 


<M*('/). '/]  = 0 
0«[*(f/)> '/]  = 0. 


(2.3) 


I 


where  q ^ «,  and  n is  the  dimension  of  the  state  vector  x(i).  It  will  be  convenient  to  write  the 
terminal  conditions  (2.3)  in  the  vector  function  form 

‘1‘lxUf),  t,\  = 0.  (2.4) 

where  we  define  the  vector  function  0 [0i t In  the  present  work  it  will  be  assumed 

that  the  functions  4>  : R*  x Rl  -*  Rl,  L : R"  x Rm  x Rl  -*  RK  f : R"  x Rm  x Rl  ->  R " and 
: RH  x R'  ■ R'i,  which  define  the  optimization  problem,  arc  continuously  differentiable  in 
all  arguments. 

2.1  Indirect  Methods 

These  methods  use  an  iterative  scheme  to  solve  some  of  the  necessary  conditions  for 
optimality,  while  satisfying  the  remaining  conditions  exactly.  Two  methods  which  have  met 
with  a degree  of  success  in  applications  are 

(a)  the  neighbouring  extremal  or  shooting  method,  and 

(/>)  the  quasi-linearization  method. 

The  necessary  conditions*  for  optimality  of  the  problem  defined  by  (2.1 )— (2.3)  are  most 
easily  stated  by  use  of  the  Hamiltonian  M(.v,  w,  A,  t)  which  is  defined  by 

W(.v,  U,  A,  l)  L(x,u.t)  I AT(/)/(je,  u,  f),  (2.5) 

where  Ml)  is  the  Lagrangian  multiplier  function.  These  conditions  give  rise  to  the  two-point 
boundary  value  problem  of  finding  functions  ,v(/),  «(/)  and  A(r).  re  [r„.  t/]  which  simultaneously 
satisfy 

dx  i H 

(i)  the  n state  equations,  (t)  (.v,  w.  A, /); 

<lt  JA 

</A  ZH 

(ii)  the  n adjoint  equations,  , (/)  — (,v.  u.  A, /); 

dr  J.v 

ZH 

(iii)  the  in  optimalitv  conditions.  0; 

Zu 

(iv)  the  ii  initial  conditions  ,v(r„)  .v„; 

(v)  (he  q terminal  conditions  </>[.v(//),  //]  0;  and 

[}<f,  ji^l 

f i,T  . where  v is  an 

> v i'i.vJ  t = If 

arbitrary  (/-vector. 


2.1.1  Neighbouring  lAtremal  Algorithms 

These  methods  which  are  also  known  as  shooting  or  root  finding  methods  in  R".  attempt 
to  solve  conditions  (i)  (iii)  exactly  by  direct  integration  of  the  differential  equations,  while  using 
an  iterative  procedure  to  converge  to  the  satisfaction  of  conditions  (iv)-(vi),  as  discussed  in 
References  6 and  7. 

In  the  case  where  condition  (iii).  above,  can  be  explicitly  solved  for  u{t)  in  terms  of  v(/ ) 
and  A( t ),  this  variable  can  be  eliminated  from  the  differential  equations  given  in  (i)  and  (ii)  above. 
Adjoining  vectors  v(r)  and  A(r)  to  form  the  2/i- vector 


'(O 

_A(/)J ' 


it  can  easily  be  seen  (hat  the  equations  given  in  (i)  and  (ii)  above,  with  u(t)  eliminated,  take  the 
form  of  the  two-point  boundary  value  problem 

t The  transpose  of  a matrix  A will  be  denoted  by  .4T. 


where 


dy 

dt 


(0  = F(y , i). 


(2.7) 


Fly,  r)  = 


OH 

0\ 

- iH 
ix 


( X , mx,  AH/),  r) 

(.v.  u{x,  A MO,  A,  t) 


and  the  boundary  conditions  are 


>\to) 


- 

•*('/),  1/) 

Xo 

, and 

Mto) 

A Ur)  - 

7*  ' 

— V1 

_ 

0X  0ATJ 

= 0. 


(2.8) 


For  simplicity,  in  the  remainder  of  this  discussion  it  will  be  assumed  that  x(t/)  x/,  where  x/ 

is  a given  vector.  In  this  case  the  terminal  condition  given  in  (2.8)  simplifies  to 

One  approach  to  solving  this  problem  is  to  consider  that  (2.7)  implicitly  defines  a function, 
F,  mapping  A(r0)  e Rn  into  (.*(//)  — x/)  e /?",  so  that  it  now  becomes  a problem  of  finding  a 
A(r„)  such  that  /*(A(/0))  = 0.  This  can  be  solved  by  using  the  classical  Newton-Raphson  method 
where  the  interates  are  constructed  as  follows 


A<+i(/„)  = A ,(t0)  - PH  ' F(\,U„)),  / = 0,  I,  2,  3,  ; (2.10) 

|_0A(/o)J 

assuming  that  (j/'/iA(r0)]  1 exists.  In  applying  this  method,  Bryson6  suggests  the  following 
three  approaches  for  determining  the  multiplier  matrix  [^A7dA(/0)]  1 : 

(i)  direct  numerical  differentiation  by  perturbing  each  component  of  A(r0)  in  turn,  followed 
by  integration  of  (2.7)  and  matrix  inversion  of  iF/i\{t0)', 

(ii)  unit  solutions  using  second  variation  equations,  followed  by  matrix  inversion  of 

0F/0\(to); 

(iii)  using  the  backward  sweep  method,  which  yields  [i/,/JA(f0)]  1 directly. 

Methods  (i)  and  (ii)  often  suffer  from  severe  numerical  sensitivity  because  the  matrix  if/DA(r0) 
is  often  ill-conditioned.  However,  the  third  approach  tends  to  be  much  less  sensitive  to  this 
conditioning  problem. 

The  main  disadvantage  of  this  approach  is  that  it  often  suffers  from  severe  conditioning 
problems,  where  small  changes  in  A(/0)  lead  to  very  large  changes  in  x(tf).  This  can  result  in  the 
algorithm  failing  to  converge,  unless  the  initial  trajectory  is  close  to  an  extremal  trajectory.  In 
addition  it  is  quite  sensitive  to  the  effects  of  numerical  rounding.  However,  because  of  the  quad- 
ratic convergence  properties  of  Newton's  method  it  is  quite  useful  for  conducting  parametric 
perturbation  studies  once  an  extremal  solution  has  been  found  by  some  other  means,  such  as 
the  gradient  method  to  be  discussed  subsequently. 


2.1.2  Quasi-Linearization  Algorithm 

An  alternative  approach  to  that  discussed  was  proposed  by  McGill  and  Kenneth,*  and 
Bellman  and  Kalaba.910  In  their  method,  instead  of  iterating  on  the  boundary  conditions  the 
procedure  now  iterates  on  the  entire  solution  trajectory  >(0.  I e [r0.  f/j. 

Considering  the  same  problem  as  defined  in  (2.7)— (2.9),  define  a function  space  operator, 
as 

jr(y)  = $ - F(y,  l).  (2.11) 

at 

It  can  be  seen  that  if  >(/)  is  to  satisfy  (2.7)  it  is  necessary  for  .#(.!')  = 0,  that  is  y has  to  be  a root 


• ] 


3 


of  the  operator  .f . Applying  Newton's  method  in  function  space  to  (2.11),  where  we  let 
.»'*(/)  * 8v*(f),  and  denote  the  partial  derivative  iF(y,  t)[iy  by  Fy(y, /),  yields 

0 .*(.i*  t 8vt)  F(.v*,f)  + - F^yk,l)\Syt.  (2.12) 

Consequently  the  linear  two-point  boundary  value  problem 

h'k  f'yivt.  i )8»*  —dj*.  (2.13) 

t/r  dt 

with  boundary  conditions  8.v(r„)  8.v(//)  0,  must  be  solved  for  the  iterate  £»*(/),  for 

k 0,  I,  2 This  problem  can  be  easily  solved  by  standard  methods. 

This  approach,  while  exhibiting  quadratic  convergence  near  an  extremal  solution  again 
suffers  with  the  problem  of  ill-conditioning,"  which  may  lead  to  the  algorithm  failing  to  con- 
verge unless  the  initial  trajectory  is  near  an  extremal  trajectory. 


2.2  Direct  Methods  on  Control  Function  Space 

The  direct  methods  of  computing  the  extremal  controls  for  optimal  control  problems 
largely  overcome  the  convergence  difficulties  of  the  indirect  methods.  In  these  methods  the  cost 
functional.  J.  in  (2.1),  is  minimized  directly  without  recourse  to  the  necessary  conditions,  by 
iteratively  adjusting  the  control  function  uU).  rt  [/„,  //].  These  methods,  which  are  well  suited 
to  handling  problems  without  terminal  constraints,  cannot  be  immediately  applied  to  problems 
of  the  type  defined  by  Equations  (2.1)  (2  3).  Methods  of  applying  the  algorithms  under  con- 
sideration to  this  latter  class  of  problems  will  be  discussed  in  the  next  section. 


2.2.1  First  Order  Gradient  Methods 

Applications  of  first  ordet  gradient  methods  have  been  extensively  studied.1-  19  because 
of  their  relative  simplicity  on  the  one  hand,  and  their  reliability  in  solving  a broad  class  of 
problems  w ith  an  acceptable  speed  of  convergence  on  the  other. 

Consider  the  case  where  the  terminal  constraints  (2.4)  are  not  present.  The  cost  functional, 
J.  defined  by  (2.1)  can  he  considered  as  mapping  the  control  we  Z.-[r„,  t/j  into  R'.  In  this  case 
the  gradient,  V./(w),  of  (2  1),  subject  to  the  differential  equation  constraints  (2.2)  can  be 
determined  by  expanding  (2.1)  using  a truncated  Taylor  series.  Thus  if  8u  denotes  a variation  in 
the  control  u.  we  obtain 

i'tf 

J(u  8u)  f/]  L(x,  w,  r)dt  ^x(v(f/),  f/)8v 

J '» 

’if 

[LAx.  u,  I)  8.v(r)  t LAx,  w.  t)8u{t)]dt  (2.14) 

J to 

where  8x (/)  is  the  variation  in  the  solution  of  (2.2)  caused  by  8 u(/).  The  variation  8.v(f),  I e [r0. 1/] 
is  the  solution  of  the  variational  differential  equation 

S.v(r)  fAx(t).  w(r)8.v(M  uU).  »)8w(r),  8.v(r„)  - 0.  (2.15) 

dt 

Introducing  the  adjoint  differential  equation 

(t)  frT{x(t).  iiU).  z )A< z ) Ltr(x(r).  uU),  t),  M tf)  *,Tl«'f).  If).  (2.16) 

dl 

it  can  be  shown  that 

i'tf  i'tf 

<M.x(t/),  tf]8x(t)  t I LAx,  u,  t)8x(t)dt  = I AT(r  )/M(.v,  w,  t )8u(i )dr.  (2.17) 

J lo  J lo 

Substituting  (2. 1 7)  into  (2. 14)  it  follows  that 
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f" 

J(u  f hu)  J(u)  + [AT(0/«(*,  w,  O i Lulx,  u,  t))bu(t)dt, 

J <0 

and  as  a consequence  the  gradient  V7(«)  can  be  identified  as 

VJ(u)U)  = \T(t)fu(x(t),  uU),  t)  -f  LuMl),  u (/),  /),  te[to,t/].  (2.18) 

Assuming  that  the  initial  estimate  of  the  control,  ua(  ■ ),  is  given,  and  using  the  usual  gradient 
procedure,  the  (A  f-  I )th  iterate  of  the  control,  uk.  i(  ■),  is  given  by 

utn  = uk  - <xkKJ(uk ),  (2.19) 

where  \J(uk)  is  given  by  (2.18).  Provided  that  an  appropriate  procedure  is  used  for  selecting 
the  parameters  {*k}£  0 it  can  be  shown  that  such  an  algorithm  converges  to  an  extremal  solution 
of  the  above  problem.  One  such  procedure  is  the  method  of  steepest  descent  where  the  para- 
meter a*  is  chosen  by  conducting  a one-dimensional  search  so  that 

J(uk)  ~ akTJ(uk)) 

is  minimized.  This  procedure  is  very  robust  because  at  each  iteration  step  the  value  of  the  cost 
functional,  J , must  decrease  until  a limit  is  reached. 

The  main  advantages  of  methods  using  a first  order  gradient  are: 

(1)  it  is  simple  to  program; 

(2)  it  requires  first  order  derivative  evaluations  only; 

(3)  global  convergence  to  local  minima  can  be  proved  so  that  the  algorithm  is  reliable; 

(4)  the  algorithm  coverges  rapidly  during  the  initial  iterations. 

However,  these  methods  exhibit  the  significant  disadvantage  of  slow  convergence  once  the 
iterates  are  in  the  neighbourhood  of  an  extremal  solution.  In  fact  it  can  be  shown  that  they  only 
have  a linear  rate  of  convergence  near  a solution  so  that  the  convergence  error  only  decreases 
with  a geometric  progression  on  the  convergence  factor. 


2.2.2  Second  Order  Gradient  Methods 

Because  of  the  poor  speed  of  convergence  of  first  order  gradient  methods  many  researchers 
have  studied  second  order  gradient  methods.-’0'21'22  '-3  24  Essentially  these  methods  involve 
expansion  of  the  cost  functional  (2.1)  to  include  quadratic  terms,  thus  leading  to  a linear  quad- 
ratic optimization  problem,  which  has  to  be  solved  for  the  new  control  iterate. 

Again  consider  the  case  where  the  terminal  constraints  (2.4)  are  not  present.  Following 
Miele18  a multiplier  function  A(r).  re  [r0,  //]  is  introduced  which  satisfies  (2.16)  and  is  used  to 

(I  V 

augment  the  functional  (2.1)  with  the  term  AT(/)(  AMO,  «(0.  0 - ' (0).  thus  yielding  the  new 

dt 

functional 


J(u)  = 4>[x(tf),  if] 


f" 

+ L(x,  u, 
J to 


0 + \T(t)(f(x,  u.t)  — d*  (r))dt, 
dt 


HxUr),  //]  + 


tf 


dx 


H(x , u.  A,  i)  - \T(,)  ■ (2.20) 

dt 


where  H(x.  u.  A,  i)  is  defined  in  (2.5).  Expanding  (2.20)  using  a truncated  Taylor  series  including 
quadratic  terms,  we  obtain 


J(u  -t  8 u)  = J(u)  f <M-v(A).  tf)Sx(lf)  + 


C'f 


[Hu( x,  u.  A,  t)hu(t)  + Hj(x,  u , A,  t )8.v( t ) 


^ &x(t)]dl  f J8 x(if)T<f>xAx(tf).  tf)8x(if) 


J" 


l [8xr(,)8ur(,)] 


Hjk(x,u,A,t)  Hxu(x,u,\,t) 
H„x(x,u.\,t ) Huu(x,u.X,t) 


MO 

MO 


dt.  (2.21) 


where  8v(/).  i e [fu.  h]  is  the  solution  of  (2.15).  It  follows  from  (2.17)  that  (2.21)  becomes 


J[u  ■ f>u)  J(u) 

'1/ 

'i/ 

//« 

Hju 

6x 

Hubuih  • 

J 

|8.vr8uT] 

% 

to 

% 

to 

Hu X 

Huh 

Su 

( i(8.v^^.v],  „.  (2.22) 

Appl>ing  the  Newton-Raphson  method  to  (2.22)  the  problem  becomes  one  of  finding  a 
Sm<(/),  i e [r„,  //]  which  minimizes  the  quadratic  cost  functional  (2.22)  subject  to  the  linear 
variational  deferential  equation  (2.15).  Once  Su,  is  computed,  the  next  control  iterate  iq,t  is 
determined  from  uni  Ui  • 8i/<.  and  the  process  then  repeated  in  a manner  similar  to  the  first 
order  gradient  procedure. 

The  necessary  conditions  for  optimality  when  applied  to  (2.22)  and  (2.15)  yield  the  following 
two  point  boundary  value  problem 


w here 


and 


d . i 
. 8.v 
ih 

•4(0 

B(i) 

£.v 

,1 

8.\ 

At 

on 

4T(r ) 

8 A 

r(D 


v(f) 


S.v(r0)  0 


8A(f/)  I(/)S.\(I/)) 


12.23) 


) j 

^ I /«//. 

»u  H u X* 

(2.24) 

Bit) 

/ If  Mu  U 

\fur. 

(2.25) 

H 

r,  HXJ 

Huu  'Hux, 

(2.26) 

r(i) 

fuHuu 

VuT. 

(2.27) 

(f) 

H XU  H uu 

1 HuT. 

(2.28) 

The  functions  in  equations  (2.24)  (2.28)  are  assumed  to  be  evaluate  along  ,v(r),  and  w(r),  for 
[r„.  tf\. 

Two  main  approaches  have  been  proposed  for  solving  the  above  two-point  boundary  problem. 
Breakwell-0  and  Kelley  -1---,  by  determining  a set  of  n linearly-independent  solutions  to  the  In 
differential  equations  (2.23)  such  that  each  solution  satisfies  the  final  boundary  conditions, 
have  used  the  principle  of  super-position  to  find  a solution  w hich  also  satisfies  the  initial  boundary 
conditions.  Unfortunately  (his  method  can  suffer  from  severe  ill-conditioning  problems;  a fact 
which  led  to  the  development  of  the  sweep  methods  proposed  by  Jacobson,22,25  McReynolds26,27 
and  Milter21  which  tends  to  overcome  this  problem. 

The  main  advantages  of  this  method  are 

(1)  The  step  size  of  hu  is  automatically  determined,  so  that  a one-dimensional  search  is  not 
required  at  each  algorithm  iteration. 

(2)  The  algorithm  exhibits  quadratic  convergence  properties  near  an  extremal  trajectory, 
and  as  a consequence  will  converge  much  more  rapidly  to  a final  solution  than  the  first 
order  gradient  method. 

However,  there  are  two  disadvantages  which  limit  its  usefulness.  These  are 

(1)  The  algorithm  may  not  always  converge,  particularly  if  the  initial  trajectory  is  far  from 
the  extremal  trajectory.  In  fact  for  this  algorithm  to  be  applicable  it  is  necessary  for 
Htl i,  to  be  positive  definite  for  ii  [r„.  //].  which  is  difficult  to  ensure  unless  the  initial 
trajectory  is  near  a minimal  trajectory.  As  a consequence  it  is  often  necessary  to  initially 
use  an  alternative  algorithm,  such  as  the  first  order  gradient  method,  as  a preliminary 
to  its  use. 

(2)  Programming  and  problem  preparation  requires  much  greater  effort  compared  with 
first  order  gradient  methods,  because  of  the  large  number  of  second  derivatives  which 
need  to  be  evaluated. 


2.2.3  Conjugate  Gradient  Methods 

The  application  of  the  conjugate  gradient  algorithm  to  optimal  control  problems  was 
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proposed  by  Sin  not  and  1 uenberger-".  Lasdon,  Miller  and  Waren.'-»  and  Pagurek  and  Woodside*0 
about  l%7  More  recently  Hestenes31  lias  discussed  Us  use  with  his  "Method  of  Multipliers" 
for  solving  constrained  problems,  while  Miele3-  has  considered  its  use  in  conjunction  with 
his  method  of  constraint  restoration. 

I he  method  is  essentially  a generalization  of  the  conjugate  gradient  algorithm  of  Fletcher 
and  Reeves33  to  function  space  problems.  Supposing  the  gradient  VT(u)  of  the  control  u,  denoted 
by  mu)  is  computed  as  m Section  2.2  I above,  and  that  the  initial  estimate  of  the  control.  u0.  is 
chosen  arbitrarily  The  algorithm  proceeds  as  follows: 

( 1 ) let  ga  g{u„)  and  set  </,.  g0; 

(2)  find  % j,  such  that  7(u,  «/,  | is  minimized: 

(2)  set  u,  \ it,  >,</,; 

(4)  lind  gi  i g(u , i ).  and  fit  ( g, . i . g, . 1 1 (g(.  g, ).  w here  (g,.g/)  is  defined  by 

i*// 

(gi.jf/)  I giT(i  lg,(t  Uh: 

J 'a 

(5)  set  d,  i gf  i fiitlt : 

(bi  Return  to  step  (’I  until  algorithm  has  converged  to  problem  solution. 

It  lias  been  shown  in  Relerence  2')  that  not  only  does  this  algorithm  exhibit  many  of  the 
theoretical  properties  ol  the  finite  dimensional  conjugate  gradient  algorithm,  but  numerical 
experience  shows  that  it  converges  more  rapidly  than  first  order  gradient  algorithms. 

The  principal  advantages  of  this  method  arc: 

(</>  The  algorithm  only  requires  the  evaluation  of  first  order  derivatives: 

(b)  It  appears  to  exhibit  quadratic  convergence  near  an  extremal  trajectory,  although  some 
doubt  about  this  is  expressed  in  Reference  1 1. 

(<  ) Programming  effort  required  is  only  a moderate  increase  over  the  first  order  methods. 

The  main  disadvantages  are 

(</)  A significant  amount  ol  computation  time  is  required  to  evaluate  the  conjugate  direc- 
tions in  function  space. 

(b)  The  algorithm  is  quite  sensitive  to  numerical  rounding,  which  means  that  some  com- 
putations need  to  be  performed  in  double  precision. 

(i  ) A one-dimensional  search  is  required  at  each  iteration,  which  can  consume  a significant 
amount  of  computation  time. 

(</)  The  algorithm  may  fail  to  converge  if  the  initial  estimate  of  the  trajectory  is  too  far  from 
an  extremal  trajectory  . As  a consequence  a first  order  gradient  method  may  be  needed 
to  obtain  a starting  solution  for  the  algorithm. 


2.2.4  Variable  Metric  Methods 

Variable  metric  methods  were  initially  developed  for  minimization  of  functions  of  a finite 
number  of  variables.  The  best  known  of  these  being  the  Davidon  algorithm,  was  subsequently 
refined  by  Fletcher  and  Powell31  and  has  become  known  as  the  Davidon- Fletcher- Powell  (DFP) 
method.  The  application  ol  this  method  to  function  space  minimization  problems  is  discussed 
in  References  25,  26.  21.  27.  Although  only  a limited  amount  of  computational  experience  with 
the  Of  P method  has  been  reported, 31,33  this  work  shows  that  its  speed  of  convergence  is  superior 
to  the  conjugate  gradient  method. 

The  Davidon- Tlcli  her- Powell  Method 

| . 

Suppose  the  gradient  V./(iz).  denoted  by  g(u).  is  computed  as  in  Section  2.2.1,  and  the  initial 
estimate  ol  the  control.  t/„,  is  chosen  arbitrarily.  The  algorithm  proceeds  as  follows: 

( 1 ) let  go  g(tio)  and  set  d„  g„; 

(2)  find  » y i such  that  ./{lit  xdi)  is  minimized: 
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(7)  return  to  step  (2)  k times; 


(8)  return  to  step  (I)  until  algorithm  has  converged  to  problem  solution. 
In  the  above  algorithm  the  inner  product  (.v,  y)  is  defined  as 


(v,  y) 


V l 

xTU  )\it)dt. 

«0 


ri 
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Steps  (7)  and  (8)  in  the  algorithm,  where  it  is  re-started  after  every  k iterations,  have  been 
introduced  to  overcome  the  difficulties  of  excessive  storage  requirements  and  the  large  increase 
in  computation  time  per  iteration  as  k increases.  In  fact  to  compute  d(.\  it  is  necessary  to  store 
2 (/  -f  2)  functions.  The  experience  of  Tripathi37  indicates  that  k should  lie  in  the  range  5-12, 
while  Pierson38  has  found  evidence  that  selecting  A-  to  be  3 or  4 actually  enhances  the  convergence 
rate. 

Apart  from  the  DKP  method  discussed  above,  other  variable  metric  algorithms  whose 
application  to  the  computation  of  optimal  controls  has  been  studied  are  the  Davidon  “Rank- 
One"  method  by  Garg39  and  the  Broyden  quasi-Newton  algorithm  by  Edge  and  Powers.40 
Both  of  these  methods  have  similar  memory  storage  requirements  to  the  DFP  method. 

The  “rank-one"  method,  discussed  by  Garg,  has  a unique  feature  of  not  requiring  a one 
dimensional  search  at  each  iteration.  Since  a considerable  amount  of  computation  time  is  often 
consumed  in  these  searches,  there  is  a possibility  of  speedier  convergence  when  this  method  is 
applied  to  optimal  control  problems.  His  experience,  albeit  on  a simple  problem,  indicates  that 
this  may  be  the  case. 

Edge  and  Power's  study  on  the  Space  Shuttle  ascent  trajectory  optimization  is  the  only 
example,  of  which  the  author  is  aware,  where  one  of  these  methods  has  been  applied  to  a realistic 
aerospace  problem.  Their  experience  shows  that  the  Broyden  algorithm  (which  is  closely  related 
to  the  DFP  method)  can  be  successfully  used  for  studying  these  types  of  problems  although 
they  make  no  attempt  to  compare  its  efficiency  with  other  types  of  algorithms. 

Since  only  limited  experience  of  the  use  of  these  methods  has  been  reported  our  conclusions 
are  to  some  extent  tentative.  However,  reported  experience  with  the  finite  dimensional  algorithm 
plus  analytical  results  leads  to  the  following  observations — 

(1)  Only  first  order  derivatives  are  required. 

(2)  Algorithm  exhibits  quadratic  convergence  near  an  extremal  trajectory.  Experience  with 
finite  dimensional  algorithms  indicates  that  this  method  converges  more  reliably  than 
the  conjugate  gradient  methods. 

(3)  Analytical  studies  show  that  the  method  starts  like  a first  order  gradient  method  and 
gradually  becomes  like  a Newton  method. 

(4)  Experience  with  finite  dimensional  problems  shows  that  it  is  less  sensitive  to  numerical 
rounding  than  the  conjugate  gradient  method. 

(5)  In  References  37,  38,  39  it  is  reported  that  these  methods  converge  more  rapidly  than 
other  comparable  methods. 

Their  most  significant  disadvantages  are: 

(I)  They  require  considerable  memory  storage  when  compared  with  first  order  gradient 
and  conjugate  gradient  methods. 
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(2)  A considerable  amount  of  computation  time  must  be  used  in  computing  the  variable 
metric  operator. 

2.2.5  Balakrishnan  Epsilon  Method 

T his  method,  proposed  b\  Balakrishnan,41  differs  from  the  direct  methods  discussed  above, 
in  respect  of  the  state  variable  v(/)  which  is  here  treaied  as  an  independent  variable  rather  than 
as  being  causally  dependent  on  the  control  u(/)  through  the  differential  equation  (2.2)  To  ensure 
Equation  (2.2)  is  satistied  he  proposed  that  it  be  adjoined  to  the  cost  functional  (2.1)  by  use  of 
a penalty  function.  Thus  (2.1)  becomes 

ft/ 

/.(.v,  u)  \ ■ /.(  v,  u,  l )dl 

J to 

where  ||-||  denotes  the  Euclidean  norm. 

By  taking  a sequence  of  n,  i 1,2 where  t/  • 0 as  / <•  oo,  and  minimizing  7,(  for 

each  i 1 . 2,  ...  it  can  be  shown  that  the  trajectory  and  control  approach  a solution  of  the 
problem  defined  by  (2.1)  (2.3).  It  has  been  found42,43  that  the  convergence  of  this  method  is 
not  particularly  sensitive  to  the  value  of  «.  and  as  a consequence  only  a small  number  of  values 
oft  need  to  be  chosen  for  practical  application. 

The  technique  used  for  minimizing  (2.29)  in  References  42  and  43  is  to  transform  this 
functional  using  the  Rayleigh-Ritz  method,  and  then  solve  the  resulting  finite  dimensional 
minimization  problem  using  the  Newton-Raphson  algorithm.  Their  experience  seems  to 
indicate  that  it  is  a method  worthy  of  further  investigation  w hen  multiple  slate  space  and  control 
constraints  are  present.  Taylor  and  Constantinides44  found  the  indirect  relationship  between 
the  error  in  the  satisfaction  of  (2.2)  and  the  error  in  the  satisfaction  of  the  terminal  constraints 

(2.3)  made  it  difficult  to  gain  an  insight  into  the  convergence  behaviour  of  the  algorithm.  This 
proves  to  be  a significant  disadvantage. 

2.3  Discussion 

It  can  be  observed  from  the  preceding  sections  that  no  single  method  exhibits  all  the  charac- 
teristics of  an  ideal  algorithm.  As  a consequence  it  is  useful  to  examine  them  from  the  viewpoint 
of  how  the  strengths  of  one  algorithm  may  be  used  to  complement  the  weaknesses  of  others. 

Since  the  first  order  gradient  methods  reliably  converge  to  extremal  trajectories,  even  though 
they  have  poor  terminal  convergence  behaviour,  it  appears  that  they  are  well  suited  for  computing 
the  starting  trajectories  for  methods  which  are  not  globally  convergent,  but  have  rapid  terminal 
convergence.  In  this  case  no  attempt  would  be  made  to  obtain  complete  convergence  using  a 
first  order  gradient  method;  instead  it  would  be  used  to  execute  a small  number  of  iterations 
before  transferring  to  a more  rapidly  convergent  algorithm. 

Of  the  techniques  discussed,  the  variable  metric  methods  seem  to  offer  the  greatest  potential 
for  general  purpose  use.  Computational  experience,  particularly  with  finite  dimensional  prob- 
lems, has  shown  them  to  be  globally  convergent  for  a broad  class  of  problems,  even  though  this 
fact  has  not  been  universally  proved  by  analytical  means.  In  addition  these  methods  exhibit 
excellent  terminal  convergence  behaviour.  Experience  in  use  has  also  shown  them  to  have  a 
superior  convergence  rate,  and  to  be  less  sensitive  to  numerical  rounding  errors,  than  the  conjugate 
gradient  methods.  One  significant  disadvantage  of  the  variable  metric  methods,  and  something 
which  is  not  apparent  for  the  finite  dimensional  case,  is  the  large  memory  storage  require- 
ments when  they  are  applied  to  function  space  problems.  In  cases  where  memory  storage  is  at 
a premium  it  may  be  necessary  to  use  second  order  gradient  or  conjugate  gradient  tnethods 
as  their  memory  requirements  are  quite  modest. 

3.  ALGORITHMS  FOR  PROBLEMS  WITH  TERMINAL  CONSTRAINTS 

In  Sections  2.2.1  2.2.4  methods  for  solving  unconstrained  optimization  problems  of  the 
type  defined  by  equations  (2.1)  and  (2.2).  but  without  the  presence  of  the  terminal  constraints 

(2.3) .  were  examined.  In  this  section  a number  of  techniques  are  discussed,  which  can  be  used  in 
conjunction  with  the  above  methods  for  solving  problems  with  terminal  constraints. 


dx 


( < ) /(.v,  u,  t)\\-dt% 


(2.29) 
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3.1  Penalty  Function  Methods 

In  the  penalty  function  method  discussed  by  Kelley,14  an  optimization  problem  with  ter- 
minal constraints  is  transformed  into  one  without  constraints.  In  applying  this  method  to  the 
problem  defined  by  (2.1)  (2.3)  the  terminal  constraints  (2.3)  are  ignored.  However,  to  ensure 
that  the  constraints  (2.3)  are  satisfied  a new  cost  functional,  Jk(u),  is  defined  by  adjoining  a term 
proportional  to  the  magnitude  of  the  constraint  error,  i P[.x(t/),  //],  to  the  cost  functional  (2.1), 
thus  giving 


Jk(h)  ^('(f/), //]  I 


If 


L(x , u,  1 )dt 


to 


mxu/i  f/]A'0[.v(//),  //]. 


(3.1) 


where  A is  a positive  definite  diagonal  penalty  function  weighting  matrix. 

By  taking  a sequence  of  matrices  Aj,  / 1,2 where  ||Aj||  ► ao  as  i * oo,  and  mini- 

mizing (3.1)  for  each  Aj,  subject  to  (2.2)  using  one  of  the  methods  given  in  Section  2.2,  it  can  be 
shown  that  the  corresponding  sequence  of  trajectories  and  controls  converge  to  a solution  of  the 
problem  defined  by  (2.1)  (2.3).  Schemes  for  adjusting  the  elements  of  the  weighting  matrix  K 
after  each  cycle  of  the  algorithm  are  discussed  by  Kelley,14  and  Moyer  and  Pinkham.16 

F.xperience  with  this  method18,45,47  has  shown  that  it  can  lead  to  failure  of  convergence  when 
using  an  optimization  algorithm  which  is  otherwise  reliable.  The  difficulty  appears  to  partially 
arise  from  rounding  errors,  as  the  magnitude  of  the  elements  of  matrix  A approach  infinity, 
causing  the  penalty  function  to  dominate  the  original  cost  function.  Also  for  finite  dimensional 
problems,  where  penalty  functions  are  used,  it  is  well  known  that  when  the  elements  of  A'  are 
large  the  augmented  cost  functional  often  has  long  narrow  ravines,  which  lead  to  slow  con- 
vergence to  the  final  solution.  Similar  difficulties  appear  to  manifest  themselves  for  function  space 
problems. 

In  spite  of  these  difficulties  the  ease  of  application  of  the  penally  function  method  has  meant 
that  it  has  been  widely  used18,38,a9,,-1,14,17,4fl,4U  with  many  of  the  computational  algorithms 
discussed  above. 
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3.2  Shifting  Boundary  Method 

Moyer  and  Pinkham18  proposed  a slight  variation  of  the  penalty  function  method  using  the 
idea  of  successive  approximation  which  they  found  gave  more  reliable  results.  In  this  method 
the  penalty  function  weighting  matrix  A'  is  held  constant  while  the  cost  functional 


Jk(u)  <A[  v( //).  //] 


If 


L(x,  u,  l )<lt  | .((</<[  v(//).  //]  | c)7’A'(i/<[.v(r/),  I/]  | <■) 


(3.2) 


.1  l„ 

is  minimized  subject  to  (2.2),  where  the  (/-vector  c is  introduced  to  shift  the  terminal  constraints. 
The  ith  iteration  of  this  vector,  denoted  a,  is  defined  recursively  by  the  relation 


('(  i'i  i V't-Vf  t(f/).  if]- 


(3.3) 


Noting  that  ,vj  i(//)  is  a function  of  a i it  follows  that  the  algorithm  will  converge  providing 
that  the  term  </i[xi  i (//),  //]  defines  a contraction  mapping.  This  method  does  not  appear  to  have 
been  extensively  studied  in  the  literature,  since  Moyer's  early  work,  although  it  has  some 
similarities  to  the  gradient  projection  method  to  be  discussed  below. 


3.3  The  Augmented  Penalty  Function  Method 

An  alternative  approach  for  improving  the  penalty  function  method,  called  variously  the 
Method  of  Multipliers  and  the  Method  of  Augmented  Penalty  Functions,  has  been  presented 
by  Hestencs.47  A number  of  variants  of  this  method  have  also  been  proposed  by  Tripathi  and 
Narendra,48  O'Doherty  and  Pierson,49  Connor  and  Vlach,50  and  Connor  and  Saltavarcas.51 

In  the  method  proposed  by  Hcstcncs  the  cost  functional  (2.1)  is  augmented  with  a quadratic 
and  a linear  penalty  function  of  the  terminal  constraint  functions  (2.3),  thus  yielding 
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mu.  A,  A)  = 4>{x(tj),  t,)  + L(X,  u,  DJI  + + WT[x(tf),  tfWl x(r,),  1,1  (3.4) 

J >o 

where  A is  a diagonal  positive  definite  penalty  function  weighting  matrix  and  A is  a ^-vector 
multiplier  which  is  to  be  determined.  The  matrix  A is  chosen  and  held  fixed  throughout  the 
computation.  The  method  proceeds  by  selecting  a A0  and  then  minimizing  mu,  A0,  A)  with  respect 
to  u,  thus  giving  u0 . In  general,  given  the  multiplier  A„  the  next  estimate  is  determined  from 

ABti  = A„  4 Ai/i[x„(tf),  1/ ],  (3.5) 

after  which  u„n  is  determined  by  minimizing  mu,  Ah+i,  A). 

Since  the  problem  defined  by  the  augmented  cost  functional  (3.4)  and  the  differential 
equation  (2.2)  has  no  terminal  constraints  the  optimization  algorithms  discussed  in  Section 
2.2  can  be  used  for  minimizing  mu.  A,  A).  From  the  limited  amount  of  experience  reported  it 
appears  to  be  a reliable  method.  Connor  and  Saltavareas51  claim  that  their  variant  of  the  basic 
algorithm  appears  to  give  superior  computational  performance  to  all  the  others  of  this  class 
which  they  have  tested. 


3.4  Gradient  Projection  Technique 


The  application  of  the  gradient  projection  technique  to  optimal  control  problems  with 
terminal  constraints  was  proposed  by  Kelley,14  and  Bryson  and  Denham.15  More  recently 
Leese52  has  presented  a generalization  of  this  method  which  is  also  applicable  to  optimization 
problems  of  the  type  defined  by  (2.IH2.3).  Experience  with  this  method  has  shown  that  it 
functions  well  for  problems  with  linear  constraints  but  is  often  unreliable  when  the  constraints 
are  non-linear.  Recent  work  on  improving  its  performance  when  used  with  non-linear  constraints 
has  been  carried  out  by  Kelley  and  Speyer,53  Kelley,  Lefton  and  Johnson,54  and  Rosen  and 
K reuser. 55 

The  essential  idea  of  the  gradient  projection  method  is  to  minimize  the  variation 
8J  = J(u  4-  8 u)  — J(u)  of  the  cost  functional  (2.1),  due  to  a variation  8 u of  the  control  function. 
This  minimization  is  carried  out  subject  to  satisfying  (2.2)  and  (2.3)  to  first  order,  and  in  addition 
satisfying  a quadratic  integral  constraint  on  the  control  variation  8u.  The  latter  condition  is 
introduced  to  ensure  that  the  problem  has  a meaningful  solution.  If  it  is  assumed  that  the  state 
trajectory  x(r),  re  [tD,  t,}  satisfies  the  boundary  conditions  a control  iterate  8u(t),  te  [f0,  if]  is 
sought  so  as  to  minimize 


8J  = 


[\T(t)fu(x,  u,t) 


to 


Lu(x,  u,  t)]8u(r)c/r, 


(3.6) 


subject  to 


f 8x(r)  =fx(x(t),  u(t ),  r)8x(t)  +fu(x(t),  u(r),  r)8u(t),  8x(t0)  = 0,  (3.7) 

at 


<px[x(tf),  t,]8x(t,)  = 0,  (3.8) 

and 

'if 

l 8ur  (t)8u(t)dt  = 1,  (3.9) 

J >0 

where  A(r)  is  the  solution  of  (2.16).  This  is  a standard  linear  quadratic  optimization  problem 
whose  solution  is  presented  in  Reference  6. 

After  each  iteration  of  the  control  function  it  is  necessary,  if  the  terminal  constraint  function 
i/>[; r(//),  //]  is  non-linear,  to  apply  a constraint  restoration  procedure.  Bryson  and  Ho6  proposed 
the  simple  approach  of  combining  the  restoration  procedure  with  the  basic  gradient  algorithm 
using  the  method  of  successive  approximation.  In  this  case  (3.8)  is  replaced  by 


<h[xUf).  <f]8x(l,)  = -€<l>[x(tf),  I,], 


(3.10) 


where  the  parameter  c < (0,  1 1.  This  method  of  constraint  restoration  should  be  compared  with 
the  shifting  boundary  method  discussed  above,  and  also  the  combined  gradient-restoration 
algorithm  of  Miele.18 

More  recently  Kelley  and  Speyer53  have  presented  a gradient  projection  version  of  the 
Davidon-Hetcher-Powell  algorithm,  which  has  been  found  to  give  good  performance  for  finite 
dimensional  optimization  problems,  when  the  constraints  are  linear.  In  order  to  improve  the 
performance  for  non-linearly  constrained  optimization  problems  Kelley  el  al.iA  have  introduced 
the  curvilinear  projection  version  of  the  Davidon  method  which  appears  to  give  a further 
improvement  in  performance  over  the  previous  methods. 

3.5  Constraint  Restoration  Methods 

The  need  for  a constraint  restoration  procedure  has  been  previously  mentioned  in  relation 
to  the  use  of  the  gradient  projection  method  with  non-linear  constraints.  The  work  of  Miele18'3- 
and  his  collaborators  has  led  to  a family  of  optimization-restoration  algorithms  which  ensure 
constraint  satisfaction.  Moyer45  has  also  proposed  an  algorithm  which  combines  an  optimiza- 
tion phase  with  a constraint  restoration  phase.  In  his  approach,  instead  of  the  cost  functional 
being  minimized  it  is  treated  as  an  additional  terminal  constraint. 

The  Mich ■ Algorithm 

I ach  cycle  of  the  sequential  gradient-restoration  algorithm  of  Miele18  consists  of  two  phases. 
Supposing  that  the  terminal  constraints  are  satisfied  at  the  beginning  of  a cycle,  then  in  the 
first  phase  one  step  of  an  unconstrained  optimization  procedure,  such  as  described  in  Section 
2.2.  is  used  to  decrease  the  cost  functional  (2.1).  In  Miele's  work  he  has  concentrated  on  the 
use  of  the  first  order  gradient  methods  for  this  purpose.  Since  in  this  first  phase  no  account  is 
taken  of  the  terminal  constraint  requirements  it  is  likely  that  condition  (2.3)  is  violated.  The 
second  phase  consists  of  adjusting  the  control  determined  in  phase  one  so  that  the  terminal 
constraints  are  satisfied  by  minimizing  the  cost  functional 

CM  J ^r[.v(//). //l#v(//). //],  (3.11) 

subject  to  the  differential  equation  (2.2).  After  this  phase  is  completed  a new  algorithm  cycle 
begins.  As  the  cost  functional  (2.1)  does  not  appear  in  (3.1 1)  explicitly,  care  needs  to  be  taken, 
by  adjusting  the  iteration  step-size  in  phase  one,  to  nsure  that  the  algorithm  will  converge  to  a 
solution  of  the  problem  defined  by  (2.1)  (2.3).  Miele's  experience  has  shown  that  this  can  easily 
be  done. 

A large  number  of  variations  of  this  basic  algorithm  are  described  by  Miele  in  Reference  32, 
where  an  extensive  bibliography  to  this  work  is  given. 

The  Mover  Algorithm 

The  algorithm  described  by  Moyer45  can  be  best  illustrated  by  examining  the  problem 
defined  by  (2.1)  (2.3).  where  it  is  assumed  that  /.(.v.  ti.  t ) = 0.  so  that  the  cost  functional,  denoted 
by  JA(u).  becomes 

JA(u)  <b[\Uf),  t,\.  (3.12) 

In  addition  it  will  be  assumed  that  the  terminal  constraints  have  the  simple  form 

.vi(M  vi /.  / I.  . . . </  n I.  (3.13) 

which  will  be  denoted  in  vector  form  by  v( If)  .f/. 

In  the  first  phase  the  problem  defined  by  (3.12).  (2.2)  and  (2.3)  is  solved  using  ^penalty 
function  approach,  where  the  penalty  function  weighting  matrix  A',  in  (2.30),  is  held  fixed.  This 
yields  an  estimate  of  the  optimal  value  of  the  cost  functional,  which  will  be  denoted  by  J*.  In 
the  second  phase  of  the  algorithm,  where  the  terminal  constraints  are  restored,  the  cost  functional 
(3.12)  is  treated  as  an  additional  terminal  constraint  by  setting 

<A[v(//).f/l  J*.  (-■'•14) 

and  then  seeking  to  minimize  the  functional 

Jr(u)  J Mvlf/Url  O*  1 ('('r)  vr)T*(v('/)  */)).  (3. 1 5> 


subject  to  (2.2).  To  improve  the  estimate  of  J*  an  iterative  scheme  is  used,  where  the  (i  | l)th 
iteration,  J*  , , ,,  is  defined  recursively  by  the  relation 

J<,.n  JA(Uc)  f -rr — r jj-  CM'/)  - X/)TK(xA.lf)  — */)■  (3.16) 

J (Me)  j 

The  subscripted  variables  u,  and  ,fr  indicate  that  they  are  obtained  from  the  minimization 
of  (3.15). 

A disturbing  feature  of  the  Moyer  algorithm  is  that  it  will  only  converge  to  an  extremal 
solution  of  the  problem  defined  by  (3.12),  (2.2)  and  (2.3)  if  the  estimated  optimal  cost  J*  is 
less  than  the  true  value  of  the  cost  for  this  problem.  If  this  is  not  so  then  the  algorithm  will 
converge  to  a non-extremal  solution. 


4.  CONCLUDING  REMARKS 

An  examination  of  methods  for  computing  the  extremal  solutions  of  optimal  control 
problems,  which  are  suitable  for  handling  typical  flight  mechanics  problems,  has  led  to  the 
conclusion  that  a two  stage  computational  procedure  should  be  used.  This  is  necessary  because 
at  the  present  time  no  single  algorithm  exhibits  the  desirable  features  of  rapid  convergence  in 
the  neighbourhood  of  an  optimum  solution  on  the  one  hand,  and  reliable  convergence  from  a 
starting  solution  which  is  not  necessarily  close  to  the  optimum  on  the  other.  Experience  has 
shown  the  Bryson  first  order  gradient  projection  method  to  converge  reliably  from  an  arbitrary 
starting  solution,  and  to  have  a high  initial  rate  of  convergence.  Thus  it  is  well  suited  for  com- 
puting the  initial  solution  estimate  in  a two  stage  procedure.  This  algorithm  has  been  imple- 
mented and  will  be  the  subject  of  a subsequent  report. 

For  the  second  stage  an  algorithm  exhibiting  a high  rate  of  terminal  convergence  is  desired. 
Of  the  higher  order  algorithms  variable  metric  methods  arc  preferred  over  the  Newton  and 
conjugate  gradient  procedures  because  of  their  greater  reliability  and  speed  of  convergence. 

Arising  from  this  work  the  author  has  become  aware  of  two  approaches  to  algorithms  for 
optimal  control  problems  which  are  worthy  of  further  consideration.  The  first  relates  to  the 
Moyer  algorithm.  Possibilities  exist  to  improve  the  performance  of  this  algorithm  by  using 
variable  metric  methods  for  cost  functional  minimization,  and  improved  techniques  for  adjusting 
the  cost  functional  estimate.  In  addition  the  applicability  of  this  class  of  algorithms  to  optimal 
control  problems  involving  state  space  and  terminal  inequality  constraints  needs  to  be 
investigated. 

The  second  approach  is  of  a more  fundamental  nature  relating  to  methods  of  generating 
optimal  algorithms  for  classes  of  problems.  These  ideas  arose  from  a consideration  of  the  form 
taken  by  the  iteration  relations  (2.18)  for  gradient  methods.  The  steepest  descent  method  can  be 
considered  as  a one-step  optimization  where  the  parameter  <**  is  chosen  so  that  ./[u*  — a*V./(u*)] 
is  minimized.  A development  of  this  notion  would  be  to  choose  simultaneously  the  parameters 
so  as  to  minimize  7[m*  — a*V./(«*)  . . . — attSJfUkJ)]  using  a multi-stage 

optimization  procedure.  It  is  conjectured  that  this  will  yield  an  improved  speed  of  convergence 
over  the  steepest  descent  method.  Whether  this  will  be  achieved,  or  achieved  at  excessive 
"computational  cost"  remains  to  be  determined.  The  concept  is  easily  generalized  to  variable 
metric  methods,  and  so  is  capable  of  generating  whole  families  of  new  algorithms. 
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further  research  on  optimisation  algorithms. 
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